Unbeatable Imitation* 

Peter Duersch^ Jorg Oechssler ^ Burkhard C. Schipper§ 

February 18, 2010 



Abstract 

We show that for many classes of symmetric two-player games, the simple de- 
cision rule "imitate-the-best" can hardly be beaten by any other decision rule. We 
provide necessary and sufficient conditions for imitation to be unbeatable and show 
that it can only be beaten by much in games that are of the rock-scissors-paper 
variety. Thus, in many interesting examples, like 2x2 games, Cournot duopoly, 
price competition, rent seeking, public goods games, common pool resource games, 
minimum effort coordination games, arms race, search, bargaining, etc., imitation 
cannot be beaten by much even by a very clever opponent. 
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"Whoever wants to set a good example must add a grain of foolishness to his 
virtue: then others can imitate and yet at the same time surpass the one they 
imitate - which human beings love to do. " Friedrich Nietzsche 

1 Introduction 

Psychologists and behavioral economists stress the role of simple heuristics or rules for 
human decision making under limited computational capabilities (see Gigerenzer and 
Selten, 2002). While such heuristics lead to successful decisions in some particular tasks, 
they may be suboptimal in others. It is plausible that decision makers may cease to 
adopt heuristics that do worse than others in relevant situations. If various heuristics are 
pitted against each other in a contest, then in the long run the heuristic with the highest 
payoff should survive. 

One of the heuristics in the contest may be a rational, omniscient, and forward looking 
decision rule. Even if such a rational rule is not currently among the contestants, there 
can always be a "mutation", i.e., an invention of a new rule, that enters the pool of rules. 
A heuristic that does very badly against such a rational rule will not be around for long 
as it will not belong to the top performers. Being subject to exploitation by the rational 
opponent in strategic situations would be an evolutionary liability. Consequently, we 
like to raise the following question: Is there a simple adaptive heuristic that can not be 
beaten even by a rational, omniscient and forward looking maximizer in large classes of 
economically relevant situations? 

The idea for this paper emerged from a prior observation in experimental data. In 
Duersch, Kolb, Oechssler, and Schipper (2010), subjects played against computers that 
were programmed according to various learning algorithms in an Cournot duopoly. On 
average, human subjects easily won against all of their computer opponents with one 
exception: the computer following the rule "imitate-the-best" , the rule that simply pre- 
scribes to mimic the action of the most successful player in the previous round. This 
suggested to us that imitation may be hard to beat even by forward-looking players. 

In this paper, we prove that this holds more generally. The decision heuristic "imitate- 
the-best" is very hard to beat by any other decision rule in large classes of symmetric 
two-player games that are highly relevant for economics and include games such as all 
symmetric 2x2 games, Cournot duopoly, Bertrand duopoly, rent seeking, public goods 
games, common pool resource games, minimum effort coordination games, synergistic 
relationship, arms races. Diamond's search, Nash demand bargaining, etc. 



1 



We shall consider two notions of being "unbeatable" . We call imitation "essentially 
unbeatable" if during the infinitely repeated game against the some opponent, the oppo- 
nent cannot obtain, in total, over an infinite number of periods, a payoff difference that 
is more than the maximal payoff difference for the one-period game. As a weaker notion 
we consider the concept of being "not subject to a money pump" . We say imitation is 
not subject to a money pump if there is a bound on the sum of payoff differences an 
opponent can achieve in the infinitely repeated game. 

It would be intractable to explicitly consider how all possible opponents - who may 
have any arbitrary decision rule - would play against an imitator. That is why we 
consider the toughest possible opponent against the imitator in order to obtain an upper 
bound for how much an imitator can be beaten by any possible opponent. This toughest 
possible opponent clearly is a dynamic relative payoff maximizer who maximizes the 
sum of all future differences between her payoff and the imitator's payoff. The relative 
payoff maximizer is assumed to be infinitely patient and forward looking and never to 
make a mistake. More importantly, the dynamic payoff maximizer is assumed to know 
that she is matched against an imitator. That is, she knows exactly what her opponent, 
the imitator, will do at all times, including the imitator's starting value. Finally, the 
dynamic payoff maximizer is enabled to commit to any strategy including any closed- 
loop strategy. Although these assumptions are certainly extreme, they make sure that if 
imitation cannot be beaten by this maximizer, then it cannot be beaten by any decision 
rule including dynamic absolute payoff maximization or any decision rule that is more 
myopic or less omniscient. 

Our results are as follows. We present necessary and sufficient conditions for imitation 
being subject to a money pump. The paradigmatic example for a money pump is playing 
repeatedly the game rock-paper-scissors, in which, obviously, an imitator can be exploited 
without bounds by the maximizer. The main result of this paper is that imitation is 
subject to a money pump if and only if the relative payoff game in question contains a 
generalized rock-paper-scissors game as a submatrix. 

Since the existence of a rock-paper-scissors submatrix may be cumbersome to check 
in some instances, we also provide a number of sufficient conditions for imitation not 
to be subject to a money pump that are based on more familiar concepts like quasicon- 
cavity, generalized ordinal potentials, or quasisubmodularity/quasisupermodularity and 
aggregation of actions. 

We also provide a number of sufficient conditions for imitation to be essentially un- 
beatable like exact potentials, increasing/ decreasing differences, or additive separability. 
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One such condition is that the game is a symmetric 2x2 game. To gain some intuition 
for this, consider the game of "chicken" presented in the following payoff matrix. 

swerve straight 



Suppose that initially the imitator starts out with playing "swerve". What should a 
forward looking opponent do? If she decides to play "straight" , she will earn more than 
the imitator today but will be copied by the imitator tomorrow. From then on, the 
imitator will stay with "straight" forever. If she decides to play "swerve" today, then 
she will earn the same as the imitator and the imitator will stay with "swerve" as long 
as the opponent stays with "swerve" . Suppose the opponent is a dynamic relative payoff 
maximizer. In that case, the dynamic relative payoff maximizer can beat the imitator 
at most by the maximal one-period payoff differential of 3. Now suppose the opponent 
maximizes the sum of her absolute payoffs. The best an absolute payoff maximizer can 
do is to play swerve forever]^ In this case the imitator cannot be beaten at all as he 
receives the same payoff as his opponent. In either case, imitation comes very close to 
the top-performing heuristics and there is no need to abandon such an heuristic. 

Imitate-the-best has been previously studied theoretically and experimentally mostly 
in Cournot oligopolies. Vega-Redondo (1997) shows that in symmetric Cournot oligopoly 
with imitators, the long run outcome converges to the competitive output if small mis- 
takes are allowed. Huck, Normann, and Oechssler (1999), Offerman, Potters, and Son- 
nemans (2002), and Apesteguia et al. (2007, 2010) provide some experimental evidence. 
Vega-Redondo's result has been generalized to aggregative quasisubmodular games by 
Schipper (2003) and Alos- Ferrer and Ania (2005). For Cournot oligopoly with imitators 
and myopic best reply players, Schipper (2009) shows that the imitators' long run average 
payoffs are strictly higher than the best reply players' average payoffs. 

The article is organized as follows. In the next section, we present the model and 
provide formal definitions for being unbeatable. Our main result, which provides a nec- 
essary and sufficient condition for a money pump, is contained in Section [31 Sufficient 
conditions for imitation to be essentially unbeatable are given in Section HI Section |5] 
provides sufficient conditions for imitation not being subject to a money pump. We finish 
with Section El where we summarize and discuss the results. 

^Payoffs are evaluated according to the over-taking criterion (sec below). 
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2 Model 



We consider a symmetrico two-player game (X, vr), in which both players are endowed 
with the same (finite or infinite) set of pure actions X and the same bounded payoff 
function n : X x X — > M, where 7r(x, y) denotes the payoff to the player choosing the 
first argument. We will frequently make use of the following definition. 

Definition 1 (Relative payoff game) Given a symmetric two-player game {X,7r) , the 
relative payoff' game is {X,A), where the relative payoff function A : X x X — M is 
defined by 

A{x,y) = iT{x,y) - ir{y,x). 

Note that, by construction, every relative payoff game is a symmetric zero-sum game 
since A{x,y) = —A{y,x). 

We introduce two types of players. The imitator follows the simple rule "imitate- 
the-best". To be precise, the imitator adopts the opponent's action if and only if in 
the previous round the opponent's payoff was strictly higher than that of the imitator. 
Formally, the action of the imitator yt in period t given the action of the other player 
from the previous period Xt-i is 

( xt-i if A(xt_i,yt_i) > , . 

y'=[m-. else 

for some initial action y^ ^ X. 

The second type we consider is a dynamic relative payoff maximizer. The dynamic 
relative payoff maximizer, from now on call her the maximizer, maximizes the sum of all 
future payoff differentials between her and the imitator, 

T 

DiT):=Y,H^t,yt), (2) 

t=o 

where yt is known to be given by ([T]). 

Since this sum may become infinite for T — )■ oo, we assume that the maximizer 
evaluates her strategies according to the overtaking- criterion (see e.g. Osborne and 
Rubinstein, 1994, p. 139). Accordingly, a sequence of relative payoffs {A{xt,yt)}'^o 
strictly preferred to a sequence {A(xJ, ?/^)}^q if lim^ inf ^^Q(A(xt, yt) — A(xJ,y^)) > 0§| 

^See for instance Weibull (1995, Definition 1.10). 

''General results on the existence of optimal over-taking strategies are developed in Leizarowitz (1996) 
and the literature cited therein. Here we can side-step the issue of existence because our proofs will be 
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If we used time-averaging instead of the over-taking criterion, then the maximizer 
would be indifferent between a sequence of zero relative payoffs and a sequence with 
a finite number of strictly positive relative payoffs and zero thereafter. Yet, our aim 
is to pit the imitator against a maximizer who cares even about a finite number of 
payoff advantages. If we used time discounting instead the over-taking criterion, then the 
maximizer may prefer a sequence with a large payoff at the beginning over a sequence 
with an endless cycle of small but positive relative payoffs. However, we believe that 
money pumps - even if small - are a feature of irrationality that a rational opponent 
would take advantage of. 

It is important to realize just how extreme our assumptions regarding the maximizer 
are. The maximizer is infinitely patient and forward looking and never makes a mistake. 
More importantly, she is assumed to know exactly what her opponent, the imitator, 
will do at all times, including the imitator's starting value. Although these assumptions 
are certainly extreme, they make sure that if imitation cannot be beaten by this max- 
imizer, then it can not beaten by any decision rule including dynamic absolute payoff 
maximization or any decision rule that is more myopic or less omniscient. 

Definition 2 (No money pump) We say that imitation is not subject to a money 
pump if there exists a bound M G IR+ such that for any initial action of the imitator 

yo e X, 

T 

lim supVA(xt,2/i) <M, (3) 

T-i-oo 

t=0 

where yt is given by 

That is, imitation is not subject to a money pump if it can be beaten only by a finite 
amount although the game between the imitator and the maximizer runs for an infinite 
number of periods. In some cases we can show that imitation can in fact not be beaten 
by more than the payoff differential from a single period. 

Definition 3 (Essentially unbeatable) We say that imitation is essentially unbeat- 
able if it can be beaten in total by at most the maximal one-period payoff differential, i.e., 
if M in inequality ^ is at most A := max^.^^^ A(x, y). 

constructive in the sense that (a) we construct strategies - no matter whether optimal or not - that beat 
the imitator or (b) we show that no strategy can beat the imitator. 
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As in previous studies of imitation (see e.g. Alos-Ferrer and Ania, 2005; Schipper, 
2003; Vega-Redondo, 1997), the concept of a finite population evolutionary stable strat- 
egy (Schaffer, 1988, 1989) plays a prominent role in our analysis. 

Definition 4 (fESS) An action x* & X is a finite population evolutionary stable strat- 
egy (fESS) of the game (X, vr) if 

7r(x*, x) > 7r(x, X*) for all x E X. (4) 

In terms of the relative payoff game, inequality (|4]) is equivalent to 

A{x*,x) > for all x e X. 

Already Schaffer (1988, 1989) observed that the fESS of the game (X, ir) and the 
symmetric pure Nash equilibria of the relative payoff game {X, A) coincide. 

3 A Necessary and Sufficient Condition for a Money 
Pump 

The game rock-paper-scissors is the paradigmatic example for how an imitator can be 
exploited without bounds by the maximizer. In our terminology, imitation is subject to 
a money pump. 

Example 1 (Rock-Paper-Scissors) Consider the well known rock-paper-scissors game 
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Clearly, if the imitator starts for instance with R, then the dynamically optimal strategy 
of the maximizer is the cycle P-S-R... In this way, the maximizer wins in every period 
and the imitator loses in every period. Over time, the payoff difference will grow without 
hound in favor of the maximizer. 

We can generalize Example [1] by noting that the crucial feature of the example is 
that the maximizer can find for each action of the imitator an action which yields her a 
strictly positive relative payoff. 

^In the following, we will represent symmetric payoff matrices by the matrix of the row player's 
payoffs only. 
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Definition 5 (GeneraHzed Rock-Paper-Scissors Matrix) A symmetric zero-sum game 
{X,7t) is called a generalized rock-paper-scissors matrix if for each column there exists a 
row with a strictly positive payoff to player 1. 

It should be fairly obvious that if a zero-sum game contains somewhere a submatrix 
that is a generalized rock-paper-scissors matrix, then this is sufficient for a money pump 
as the maximizer can make sure that the process cycles forever in this submatrix. What 
is probably less obvious is that the existence of such a submatrix is also necessary for a 
money pump. 

Definition 6 (Generalized Rock-Paper-Scissors Game) A symmetric zero-sum game 
(X, vr) is called a generalized rock-paper-scissors game if it contains a submatrix (X, vf ) 
with X C X and vf (x, y) = n{x, y) for all x,y ^ X , and {X, vf) is a generalized rock- 
paper-scissors matrix. 

This leads us to our main result. 

Theorem 1 Imitation is subject to a money pump in the finite symmetric game {X, vr) 
if and only if its relative payoff game (X, A) is a generalized rock-paper-scissors game. 

Recall that according to Definition [2l imitation is subject to a money pump if there 
exists some initial condition yo & X such that inequality ([3]) is violated. The proof of 
the theorem follows directly from the following three lemmata. We use the following 
preliminary observation repeatedly in the analysis. 

Lemma 1 Consider a symmetric game (X, tt) with its relative payoff game (X, A). The 
maximizer will never choose an action Xt such that A(a;t, yt) < 0. 

Proof. Suppose to the contrary that the maximizer chooses an action xt such that 
A{xt,yt) < 0. Then in period t + 1, the imitator will not imitate her period t action. 
But then, she could improve her relative payoff in t by setting the same action in t as 
the imitator without influencing the actions of the imitator in period t -|- 1 or any other 
future period. □ 

Given a symmetric two-player game (X, vr) and its relative payoff game (X, A), a path 
in the action space X x X is a sequence of action proflles {xo,yo), .... A path is 
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constant if {xt,yt) = {xt+i,yt+i) for all t = 0,1,.... Otherwise, the path is called non- 
constant. A non-constant finite path {xo,yo), {xn,yn) is a cycle if (xo,yo) = {xn,yn)- 
A cycle is an imitation cycle if for all {xt,yt) and {xt+i,yt+i) on the path of the cycle 
A{xt, yt) > and yt+i = Xt- Along an imitation cycle, one player always obtains a strictly 
positive relative payoff and the other player mimics the action of the first player in the 
previous round. Thus, an imitation cycle never contains an action profile on the diagonal 
of the payoff matrix. 

Lemma 2 For any finite symmetric game (X, vr), imitation is subject to a money pump 
if and only if there exists an imitation cycle. 

Proof. Consider a finite symmetric game (X, vr) and its relative payoff game (X, A). 
We show first that if imitation is subject to a money pump, then there is a imitation cycle. 
Since the game is finite, there can not be infinitely many strictly positive relative payoff 
improvements unless there is a cycle. To show that such a cycle implies an imitation 
cycle, suppose by contradiction that there exists a period t such that A{xt,yt) < 0. 
W.l.o.g. assume that A{xt+i,yt+i) > 0. This is w.l.o.g. because we assumed a money 
pump. By equation ([1]) the imitator will not imitate in t + 1 the previous period's action 
of the maximizer, i.e., yt+i = yt- But then the maximizer could strictly improve the sum 
of her relative payoffs already in t by setting xt = Xt+i- Thus, there must be a cycle with 
A{xt,yt) > for all t. The decision rule of the imitator then requires that yt+i = Xt for 
all t, which proves that such a cycle is an imitation cycle. 

The converse is trivial. □ 

Lemma 3 Consider a finite symmetric game {X, vr) with its relative payoff game {X, A) . 
{X, A) is a generalized rock-paper-scissors game if and only if there exists an imitation 
cycle. 

Proof. "<^=": If there exists an imitation cycle in (X, A), let X be the orbit of the 
cycle, i.e., all actions of X that are played along the imitation cycle. For each action 
(i.e., column) y G X, there exists an action (i.e., row) x G X such that A{x,y) > 0. 
Hence, (X, A), where A is defined by A{x, y) = A(x, y) for all x, ?/ G X, is a generalized 
rock-paper-scissors submatrix. Thus, (X, A) is a generalized rock-paper-scissors game. 

: If the relative payoff game (X, A) is a generalized rock-paper-scissors game, 
then it contains a generalized rock-paper-scissors submatrix (X,A). That is, for each 
column of the matrix game (X, A) there exists a row with a strictly positive relative 
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payoff to player 1. Let the initial action of the imitator y be contained in X. If the 
maximizer selects such a row x E X for which she earns a strict positive relative payoff, 
i.e., A(x, y) > 0, then she will be imitated by the imitator in the next period. Yet, at the 
next round, when the imitator plays x, the maximizer has another action x' E X with 
a strictly positive relative payoff, i.e., A{x',x) > 0. Thus the imitator will imitate her 
in the following period. More generally, for each action ?/ G X of the imitator, there is 
another action x E X, x ^ y of the maximizer that earns the latter a strictly positive 
relative payoff. Since X is finite, such a sequence of actions must contain a cycle. More- 
over, we just argued that A{xt,yt) > and yt+i = Xt for all t. Thus, it is an imitation 
cycle. □ 



Theorem [T] is used to obtain an interesting necessary condition for imitation being 
not subject to a money pump. 

Proposition 1 Let {X, vr) be a finite symmetric game with its relative payoff' game 
(X, A). // (X, A) has no pure saddle point, then imitation is subject to a money pump. 

Proof. By Theorem 1 in Duersch, Oechssler, and Schipper (2010), (X, A) has no 
symmetric pure saddle point if and only if it is a generalized rock-paper-scissors matrix. 
Thus, if (X, A) has no symmetric pure saddle point, then it is a generalized rock-paper- 
scissors game. Hence, by Theorem [1] imitation is subject to a money pump. □ 

Corollary 1 // the finite symmetric game (X, it) has no fESS, then imitation is subject 
to a money pump. 

In other words, the existence of a fESS is a necessary condition for imitation not being 
subject to a money pump. The reason for the existence of a fESS not being sufficient is 
that there could be a generalized rock-paper-scissors submatrix of the game ("disjoint" 
from the fESS profile) that gives rise to an imitation cycle. If the initial action of the 
imitator lies within the action set corresponding to this submatrix, then imitation is 
subject to a money pump. 

Since the relative payoff game of a symmetric zero-sum game is a generalized rock- 
paper-scissors game if and only if the underlying symmetric zero-sum game is a general- 
ized rock-paper-scissors game, we obtain from Theorem [T] the following corollary. 

Corollary 2 Imitation is subject to a money pump in the finite symmetric zero-sum 
game (X, vr) if and only if (X, vr) is a generalized rock-paper-scissors game. 
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4 Sufficient Conditions for Essentially Unbeatable 



4.1 Symmetric 2x2 games 

In this section, we extend the "chicken" example of the introduction to all symmetric 
2x2 games. Note that the relative payoff game of any symmetric 2x2 game cannot be a 
generalized rock-paper-scissors matrix since latter must by a symmetric zero-sum game. 
If one of the row player's off-diagonal relative payoffs is a > 0, then the other must be 
—a violating the definition of generalized rock-paper-scissors matrix. Thus Theorem [T] 
implies that for any symmetric 2x2 game imitation is not subject to a money pump. We 
can strengthen the result to imitation being essentially unbeatable. 

Proposition 2 In any symmetric 2x2 game, imitation is essentially unbeatable. 

Proof. Let X = {x,x'}. Consider a period t in which the maximizer achieves a 
strictly positive relative payoff, A(x, x') > 0. (If no such period t in which the maximizer 
achieves a strictly positive relative payoff exists, then trivially imitation is essentially 
unbeatable.) By definition, A{x,x') < A. Since A(x,x') > 0, the imitator imitates x in 
period t + 1. For there to be another period in which the maximizer achieves a strictly 
positive relative payoff, it must hold that A(x', x) > 0, which because of symmetry yields 
a contradiction as A{x',x) = — A(x, x'). Thus there can be at most one period in which 
the maximizer achieves a strictly positive relative payoff. □ 

Note that "Matching pennies" is not a counter-example since it is not symmetric. 

4.2 Additively Separable Relative Payoff Functions 

Next, we consider relative payoff functions that are additively separable in the players' 
actions. In this class of symmetric games, imitation is also essentially unbeatable. While 
additive separability may appear to be restrictive, we will show below that there is a 
fairly large number of important examples that fall into this class. 

Definition 7 (Additive Separable) We say that a relative payoff function A is addi- 
tively separable if A{x,y) = f{x) + g{y) for some functions f,g:X — y R. 

Proposition 3 Let (X, vr) be a symmetric game with its relative payoff game (X, A). // 
X is compact and the relative payoff function A is upper semicontinuous and additively 
separable, then imitation is essentially unbeatable. 
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Proof. Since A is separable, we have that for all G X, 

A{x", x) - A(x', x) = A{x'\ x') - A(x', x'), 

which is equivalent to 

A(x", x) = A(x", x') + A(x', x) (5) 
because A{x',x') = since the relative payoff game is a symmetric zero-sum game. 

By induction, equation ([5]) implies that one large step is just as profitable as any 
number of steps. Suppose three steps were optimal for the maximizer. By equation ([5]) 
the maximizer is no worse off by merging two of the three steps to one larger step. 
Applying equation ([5]) again yields the claim. 

Thus, if a fESS exist, then the maximizer cannot do better than jumping directly to 
a fESS X* since for all x,y E X, 

A{x*,y) = A{x*,x) + A(x, y) > A(x, y), 

where the equality follows from equation ([5]) and the inequality from the definition of 
fESS. If the inequality is strict, then once the maximizer has chosen x*, the imitator will 
follow and remain there for ever. Otherwise, if the inequality holds with equality then 
the maximizer can not improve further his relative payoff. 

Finally, in Duersch, Oechssler, and Schipper (2010, Corollary 6), we show that if X 
is compact and A upper semicontinuous and additively separable, then a fESS of {X, vr) 
indeed exists. □ 

The following corollary follows directly from Proposition [3] and may be useful in 
applications. 

Corollary 3 Consider a game {X, vr) with a compact action set X and a payoff function 
that can be written as 7r{x,y) = f{x) + g{y) + a{x,y) for some continuous functions 
f,g : X — y M and a symmetric function a : X x X — > M (i.e., a{x,y) = a{y,x) for all 
x,y E X). Then imitation is essentially unbeatable. 

Properties such as increasing or decreasing differences are often useful for proving the 
existence of pure equilibria and convergence of learning processes. 

Definition 8 Let X be a totally ordered set. A (relative) payoff function A has decreas- 
ing (resp. increasing) differences on X x X if for all x",x',y",y' G X with x" > x' and 



11 



y" > y', 

A(x",y") - ^{x\y") < (>)A(x",y') - ^{x\y'). (6) 
A zs a valuation if it has both decreasing and increasing differences. 

Our original intent was to study the consequences of A(x, y) having either increasing 
or decreasing differences. However, in Duersch, Oechssler, and Schipper (2010, Lemma 
1) we show that for all symmetric two-player zero-sum games, increasing differences is 
equivalent to decreasing differences. By Topkis (1998, Theorem 2.6.4.), a function is 
additively separable on a totally ordered set X if and only if it is a valuation. Hence, we 
have the following corollary to Proposition |3l 

Corollary 4 Let (X, tt) he a finite symmetric game with its relative payoff game (X, A) . 
If X is a totally ordered set and A has increasing or decreasing differences or is a valu- 
ation, then imitation is essentially unbeatable. 

Branzei, Mallozzi, and Tijs (2003, Theorem 1) show that a zero-sum game is an exact 
potential game if and only if it is additively separable. Thus, Proposition |3] implies that 
imitation is essentially unbeatable in exact potential games. The following notion is due 
to Monderer and Shapley (1996). 

Definition 9 (Exact potential games) The symmetric game (X, vr) is an exact po- 
tential game if there exists an exact potential function P : X x X — > M such that for all 
y E X and all x, x' G X , 

■K{x,y) -Txi^x' ,y) = P{x,y) - P{x',y), 
Tr{x,y) - TT{x',y) = P{y, x) - P{y, x'). 

Corollary 5 Let {X, vr) be a finite symmetric game with its relative payoff game {X, A). 
// {X, A) is an exact potential game, then imitation is essentially unbeatable. 

All of the following examples follow from Proposition |3] or Corollary |3l They demon- 
strate that the assumption of additively separable relative payoffs is not as restrictive as 
may be thought at first glance. 

Example 2 (Cournot Duopoly with Linear Demand) Consider a Cournot duopoly 
given by the symmetric payoff function by 'K[x,y) = x{h — x — y) — c{x) with b > 0. Since 
7r(x, ?/) can be written asTi{x.,y) = hx — hx^ — c{x) — xy , Corollary applies, and imitation 
is essentially unbeatable. 
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The following example with strategic complementarities shows that the result is not 
restricted to strategic substitutes. 

Example 3 (Bertrand Duopoly with Product Differentiation) Consider a differ- 
entiated duopoly with constant marginal costs, in which firms 1 and 2 set prices x and y, 
respectively. Firm I's profit function is given by 7r(x,i/) = {x — c){a-\-by — ^x) , for a > 0, 
b E [0, 1/2). Since 7r(x, y) can be written as n{x, y) = ax — ac + |cx — |a;^ — bey + bxy, 
Corollaryl^ applies, and imitation is essentially unbeatable. 

Example 4 (Public Goods) Consider the class of symmetric public good games de- 
fined by 7r{x,y) = g{x,y) — c(x) where g{x,y) is some symmetric monotone increasing 
benefit function and c{x) is an increasing cost function. Usually, it is assumed that g is 
an increasing function of the sum of provisions, that is the sum x + y. Various assump- 
tions on g have been studied in the literature such as increasing or decreasing returns. In 
any case. Corollary applies, and imitation is essentially unbeatable. 

Example 5 (Common Pool Resources) Consider a common pool resource game with 
two appropriators . Each appropriator has an endowment e > that she can invest in 
an outside activity with marginal payoff c > or into the common pool resource, x G 
X ^ [0,e] denotes the maximizer's investment into the common pool resource (likewise y 
denotes the imitator's investment). The return from investment into the common pool re- 
source is ■^^{a{x-\-y) — b{x+y)'^) , with a,b > 0. So the symmetric payoff function is given 
by 7r(x, y) = c(e — x) + ^^(a(x + i/) — b{x-\-yY) if x,y > and ce otherwise (see Walker, 
Cardner, and Ostrom, 1990). Since A{x,y) = {c{e — x) + ax — bx'^) — {c{e — y) + ay — by'^), 
Proposition\^ implies that imitation is essentially unbeatable. 

Example 6 (Minimum Effort Coordination) Consider the class of minimum effort 
games given by the symmetric payoff function TT{x,y) = min{x,y} — c{x) for some cost 
function c (see Bryant, 1983 and Van Huyck, Battalia, and Beil, 1990). Corollary 
implies that imitation is essentially unbeatable. 

Example 7 (Synergistic Relationship) Consider a synergistic relationship among two 
individuals. If both devote more effort to the relationship, then they are both better off, 
but for any given effort of the opponent, the return of the player's effort first increases 
and then decreases. The symmetric payoff function is given by 7r(x,y) = x{c + y — x) 
with c > and x,y E X C M+ with X compact (see Osborne, 2004, V- 39). Corollary\^ 
implies that imitation is essentially unbeatable. 
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Example 8 (Arms Race) Consider two countries engaged in an arms race (see e.g. 
Milgrom and Roberts, 1990, p. 1272). Each player chooses a level of arms in a compact 
totally ordered set X . The symmetric payoff Junction is given by 7r(x, y) = h{x — y) — c(x) 
where h is a concave function of the difference between both players' level of arms, x — y, 
satisfying h{x — y) = —h{y — x). By Proposition\^ imitation is essentially unbeatable. 

Example 9 (Diamond's Search) Consider two players who exert effort searching for 
a trading partner. Any trader's probability of finding another particular trader is pro- 
portional to his own effort and the effort by the other. The payoff function is given 
by TT{x,y) = axy — c{x) for a > and c increasing (see Milgrom and Roberts, 1990, 
p. 1270). The relative payoff game of this two-player game is additively separable. By 
Proposition^^ imitation is essentially unbeatable. 

A natural question is whether additive separability of relative payoffs is also necessary 
for imitation to be essentially unbeatable. The following counter-example shows that this 
is not the case. 

Example 10 (Coordination game with outside option) Consider the following co- 
ordination game with an outside option (C) for both players of not participating (left 
matrix). 

ABC ABC 
_ A /4-10\ /0-3 0\ 

^ ~ B 230 ~ B 300 

C \ J C \ J 

Note that the relative payoff game A (right matrix) does not have constant differences. 
E.g., A{A,B) - {B,B) = -3 ^ A(AC') - A{B,C) = 0. Thus, by Topkis (1998, 
Theorem 2.6.4.), ^■^ n-ot additively separable. Yet, imitation is essentially unbeatable. 
If the imitator's initial action is A, the maximizer can earn at most a relative payoff 
differential of 3 after which the imitator adjusts and both earn zero from there on. For 
other initial actions of the imitator, the maximal payoff difference is at most 0. 

5 Sufficient Conditions for No Money Pump 
5.1 Relative Payoff Games with Potentials 

Potential functions are often useful for obtaining results on convergence of learning al- 
gorithms to equilibrium, existence of pure equilibrium, and equilibrium selection. In 
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the previous section, we have shown that if the relative payoff game is an exact potential 
game, then imitation is essentially unbeatable. It is natural to explore the implications of 
more general notions of potentials. Besides exact potential games (see Definition |9]) , the 
following notions of potential games were introduced by Monderer and Shapley (1996). 

Definition 10 (Potential games) The symmetric game {X, vr) is 

(W) a weighted potential game if there exists a weighted potential function P : XxX — y 
M and a weight w G M+ such that for all y E X and all x, x' G X , 

■K{x,y) - Ti{x\y) = w{P{x,y) - P{x',y)), 
nix^y) - TT{x',y) = w {P{y, x) - P{y, x')) . 

(O) an ordinal potential game if there exists an ordinal potential function P : XxX — ¥ 
M such that for all y E X and all x, x' G X , 

7i{x, y) — 'jt{x' , y) > if and only if P{x, y) — P{x , y) > 0, 
7i{x, y) — 7r(x', y) > if and only if P{y, x) — P{y, x) > 0. 

(G) a generalized ordinal potential game if there exists a generalized ordinal potential 
function P : X x X — > R such that for all y E X and all x, x' G X , 



Note that every exact potential game is a weighted potential game, every weighted 
potential game is an ordinal potential game, and every ordinal potential game is a gen- 
eralized ordinal potential game. Monderer and Shapley (1996, Lemma 2.5 and the first 
paragraph on p. 129) show that any finite strategic game admitting a generalized or- 
dinal potential possesses a pure Nash equilibrium. Thus, if {X, vr) is a finite symmetric 
game with relative playoff game {X, A) and the latter is an exact, weighted, ordinal or 
generalized ordinal potential game, then (X, ir) possesses a fESS. 

A sequential path in the action space XxX is a sequence {xq, yo), {xi,yi), ... of profiles 
{xt,yt) G X X X such that for all t = 0, 1, the action profiles (xt.yt) and {xt+i,yt+i) 
differ in exactly one player's action. A sequential path is a strict improvement path if 
for each t = 0,1, the player who switches her action at t strictly improves her payoff. 
A finite sequential path {xo,yo), {xm,ym) is a strict improvement cycle if it is a strict 
improvement path and {xo,yo) = {xm,ym)- 



'K{x,y) 
Tr{x,y) 



'7i{x',y) > implies P{x,y) — P{x',y) > 0, 
n{x', y) > implies P{y, x) — P{y, x) > 0. 
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Lemma 4 // {X, A) does not contain a strict improvement cycle, then it does not contain 
an imitation cycled 

Proof. We prove the contrapositive. I.e., if {X,A) contains an imitation cycle, 
then it contains a strict improvement cycle. Let {xo,yo), {xm,ym) be an imitation 
cycle. From this imitation cycle, we construct a strict improvement cycle as follows: For 
t = 0,...,m — 1, we add the element {xt,yt+i) as successor to {xt,yt) and predecessor 
to {xt+i,yt+i)- That is, instead of simultaneous adjustments of actions at each round 
as in an imitation cycle, we let players adjust actions sequentially by taking turns. The 
imitator adjusts from {xt, yt) to (xf, yt+i) and the maximizer from (xj, yt+i) to (x^+i, yt+i) 
for t = 0, ...,m — L This construction yields a sequential path. 

We now show that it is a strict improvement cycle. First, for the imitator, whenever 
she adjusts in t = 0, ...,m — 1, we claim A(|/f,Xj) < A(i/(+i,Xt) = 0. Note that by 
symmetric zero-sum, A(?/t,xt) = — A(xt,?/t) < because {xt,yt) is an element of an 
imitation cycle, i.e., A{xt,yt) > 0. A{yt+i,Xt) = because the imitator mimics the 
action of the maximizer, yt+i = Xt- Thus A(?/t+i,a;f) = A(xt,Xf) = by symmetric 
zero-sum. 

Second, for the maximizer, whenever she adjusts in t = 1, m, A(x(, yt) > A(x(_i, y^) = 
because {xt, yt) is an element of an imitation cycle, so A{xt, yt) > 0. Moreover, the 
imitator mimics the action of the maximizer, i.e., yt = Xt-i, and thus A{xt-i,yt) = 
A {xt-i,Xt-i) = 0. Hence {xo,yo), {xo,yi), {xi,yi), {xm-i,ym), {xm,ym) is indeed a 
strict improvement cycle. □ 



The converse is not true as the following counter-example shows. 
Example 11 Consider the following relative payoff game% 



a 
b 
c 




^Ania (2008, Proposition 3) presents a similar result according to which if all players are imitators 
and imitation is payoff improving, then fESS implies Nash equilibrium action. This is different from 
Lemma |4] as we consider one maximizer and one imitator and focus on the relationship between relative 
payoff games that possess a generalized ordinal potential and imitation cycles. 

^This example appears also in Ania (2008, Example 2), where it is used to demonstrate that the 
class of games where imitation is payoff improving (when all players are imitators) is not a subclass of 
generalized ordinal potential games. 
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Clearly, this game is not a generalized rock-paper-scissors game. Thus, by Lemma\^it 
does not possess an imitation cycle. However, we can construct a strict improvement 
cycle {b,a), (c, a), (c, c), {b,c) and {b,a). 

Proposition 4 Let (X, vr) be a finite symmetric game with its relative payoff game 
{X, A) . // (X, A) is a generalized ordinal potential game, then imitation is not subject 
to a money pump. 

Proof. Monderer and Shapley (1996, Lemma 2.5) show that a finite strategic game 
has no strict improvement cycle (what they call the finite improvement property) if and 
only if it is a generalized ordinal potential game. Since this result holds for any finite 
strategic game, it holds also for any finite symmetric zero-sum game (X, A). 

Lemma H] shows that if (X, A) does not contain a strict improvement cycle, then it 
does not contain an imitation cycle. Thus Lemma [2] implies that imitation is not subject 
to a money pump. □ 

If the converse were true, then the class of generalized ordinal potential relative payoff 
games and relative payoff games that are not generalized rock-paper-scissors games would 
coincide. Yet, the converse is not true. This follows again from Example [TTl It is not 
a generalized rock-paper-scissors game but due to the existence of a strict improvement 
cycle it does not possess a generalized ordinal potential by Monderer and Shapley (1996, 
Lemma 2.5). 

For an example of a game whose relative payoff game is a generalized ordinal potential 
game see again the coordination game with an outside option presented in Example [101 
A generalized ordinal potential function is given by 



It is straightforward to check that exact, weighted, and ordinal potential games are 
generalized ordinal potential games. 

Corollary 6 Let (X, tt) be a finite symmetric game with its relative payoff game (X, A). 
// (X, A) is an exact potential game, a weighted potential game, or an ordinal potential 
game, then imitation is not subject to a money pump. 



ABC 



C 



A 
B 
C 
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5.2 Quasiconcave Relative Payoff Games 

Here we show that imitation is essentially unbeatable if the relative payoff game is "qua- 
siconcave". The following definition naturally extends the definition of quasi-concave 
payoff function on convex real-valued spaces to the case of finite, symmetric games. It is 
the notion of single-peakedness. 

Definition 11 (Quasiconcave) A finite symmetric game {X, n) with symmetric mxm 
pay ojf matrix vr = (nxy) is quasiconcave (or single-peaked) if for each y E X, there exists 
a ky such that 

T^ly < T^2y < ■ ■ ■ < T^kyy > T^ky+ly > ■ ■ ■> T^my 

That is, we say a symmetric game is quasiconcave if each column has a single peak. 
In our companion paper, Duersch, Oechssler, and Schipper (2010, Corollary 4), we show 
that if X is finite and A is quasiconcave, then a fESS of (X, tt) exists. 

It is clear from the definition of quasiconcavity that if A is quasiconcave then there 
exists a total order on the action space. With some abuse of notation, we denote this 
order also by <. If A is quasiconcave, we say that x is between some x' and x" if 
x' < X < x" or x" < X < x' . 

Lemma 5 Let (X, A) he the relative payoff game of the symmetric game (X, vr). Suppose 
A is quasiconcave. 

1. If X is between some y and some fESS x* , then 

A{x,y) = -A{y,x) > 0. 

2. If X* and x** are fESS, then so are all x between x* and x** . 

Proof. (1) Let x be between y and x*. By definition of the fESS, A{x*,y) > 0. By 
symmetry of payoffs, A(y, y) = 0. Part (1) of the lemma follows then by quasiconcavity. 

(2) Let X* and x** both be fESS. Thus, A{x\x**) > and A{x**,x*) > 0. Since 
A{x*,x**) = —A{x**,x*), we must have A{x*,x**) = 0. By quasiconcavity, A{x',x**) > 
and A(x', x*) > for all x' between x* and x**. By part (1) of the lemma, A(x', x) > 0, 
for all X G X. Hence, all x' between x* and x** are fESS as well. □ 

Proposition 5 Let {X, tt) be a finite symmetric game with its relative payoff game 
{X,A). If A is quasiconcave, then imitation is not subject to a money pump. 
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Proof. We will show that the imitator's play must reach the set of fESS in finitely 
many steps, which implies that imitation is not subject to a money pump. 

We need the following notation. Let E denote the (finite) set of fESS of (X, tt). By 
Lemma O (2), if x* and x** are two fESS with x* < x**, then for any x with x* < x < x** 
also X is a fESS, where the total order < is induced by quasiconcavity of A. Denote 
X* := mini?, x** := maxE the smallest and largest fESS, respectively, where again the 
max and min are taken with respect to the total order induced by quasiconcavity of A. 
We denote by < the strict part of <, i.e., x < x' if and only if x < x' and not x' < x. 

For any value y & X, y > x**, we define the following lower bound (which need not 
always exist), 

l{y) := max {x E X : A(x, y) < 0,x < x*} . 

If y is larger than the largest fESS, then l{y) is the largest action lower than the lowest 
fESS at which relative payoffs are strictly negative. Likewise, foi y & X, y < x*, we 
define the following upper bound (which need not always exist), 

u{y) := min {x G X : A(x, y) < 0,x > x**} . 

Without loss of generality, let yo > x** be the starting value of the imitator. (The case 
of yo < X* follows analogously.). Let us consider all possible choices of the maximizer. 
By Lemma dl the maximizer will never choose an action x such that A(x, ?/o) < 0- If the 
maximizer chooses any x such that A(x, yo) = 0, then the maximizer will not be imitated 
and the situation in t = 1 will be identical to t = 0. Thus, from now on we can restrict 
attention to x G X such that A{x,yo) > 0. 

We claim that A(x, yo) > can occur only if x < yo and l{yo) < x, where the second 
requirement is empty should l{yo) not exist. To see that we can exclude x < l{yo) note 
that A(/(|/o),yo) < by definition. By quasiconcavity, A{x,yo) < for all x < /(yo)- To 
see that we can exclude x > yo note that A(?/o,Z/o) = 0. By quasiconcavity, A{x,yo) < 0, 
for all X > yo- This proves the claim. 

When the maximizer chooses any x such that l{yo) < x < yo, the imitator imitates 
X and chooses yi = x in the next period. Consider the case yi < x*. We claim that 
u{yi) < yo- To see this note that A{yi,yo) > (otherwise the imitator would not have 
imitated) and hence A{yo, yi) < 0. By quasiconcavity and the definition of fESS we have 
A(x', yi) < for all x' > yo- Hence, u{yi) < yo- Next, consider the case that yi > x**- In 
that case simply restart the procedure with the new starting value yi- Finally, consider 
the case where the maximizer selects an element in E- Then no further relative payoff 
improvements are possible. 
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Thus, in this first step we have strictly narrowed down the range of the possible choice 
yi of the imitator in period t = 1 to l{yo) < yi < y^. Since X is finite, when we repeat 
this step, the imitator must reach the set of fESS in a finite number of steps. Once the 
imitator has reached a fESS, he has reached a stationary state since then A(x,x*) < 
for all X. The imitator will never leave x* and the maximizer will never again obtain a 
positive relative payoff. Since there are only finitely many rounds in which A(xt, yt) > 0, 
imitation is not subject to a money pump. □ 

The following corollary may be useful for applications. Let X C be a finite 
subset of a finite dimensional Euclidean space. A function / : X — > M is convex (resp. 
concave) if for any x,x' G X and for any A G [0,1] such that Ax + (1 — X)x' G X, 
/(Ax + (1 - A)x') < (>)A/(x) + (1 - A)/(x'). 

Corollary 7 Let (M™, vr) be a symmetric two-player game for which 7r(-, ■) is concave in 
its first argument and convex in its second argument. If the players ' actions are restricted 
to a finite subset X of the finite dimensional Euclidian space M™, then imitation is not 
subject to a money pump. 

Bargaining is an economically relevant situation involving two players. Our results 
imply that imitation is not subject to a money pump in bargaining as modeled in the 
Nash Demand game. 

Example 12 (Nash Demand Game) Consider the following version of the Nash De- 
mand game (see Nash, 1953). Two players simultaneously demand an amount in M^. 
// the sum is within a feasible set, i.e., x + y < s for s > 0, then player 1 receives the 
payoff 7T [x , y) = x. Otherwise 7i{x,y) = (analogously for player 2). The relative pay- 
off function is quasiconcave. If the players' demands are restricted to a finite set, then 
Proposition's^ implies that imitation is not subject to a money pump. 

Example 13 Consider a symmetric two-player game with the payoff function given by 
7r{x,y) = I withx,y G X C [1,2] withX being finite. This game's relative payoff function 
is quasiconcave. Thus our result implies that imitation is not subject to a money pump. 
Moreover, the example demonstrates that not every quasiconcave relative payoff function 
is additively separable. 

Finally, we like to remark that Example [11] is an instance of a quasiconcave relative 
payoff game but due to the strict improvement cycle it does not posses a generalized 
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ordinal potential. Moreover, in Duerscli, Oechssler, and Schipper (2010, Example 1) we 
show that there are relative payoff games that are neither generalized rock-paper- scissors 
games nor quasiconcave. 

5.3 Aggregative Games 

Many games relevant to economics possess a natural aggregate of all players actions. 
For instance, in Cournot games the total market quantity or the price is an aggregate. 
But also other games like rent-seeking games, common pool resource games, public good 
games etc. can be viewed as games with an aggregate. The aggregation property has 
been useful for the study of imitation and fESS in the literature (see Schipper, 2003, and 
Alos- Ferrer and Ania, 2005). In this section, we will derive results for aggregative games 
whose absolute payoff functions satisfy some second-order properties|l| 

We say that {X, U) is an aggregative game if it satisfies the following properties. 

(i) X is a totally ordered set of actions and Z is a totally ordered set. 

(ii) There exists an aggregator a : X x X — Z that is 

— monotone increasing in its arguments, i.e. if [x", y") > {x', y'), then a{x", y") > 
a{x', y'), and 

— symmetric, i.e., a{x,y) = a{y,x) for all x,y E X. 

(iii) vr is extendable to U : X x Z — y M with n(x, a(x, y)) = ir^x, y) for all x,y E X. 

We say that an aggregative game {X, 11) is quasisuhmodular (resp. quasisupermodular) 
if n is quasisuhmodular (resp. quasisupermodular) in (x, y) o\i X x Z, i.e., for all z" > z', 
x" > x', 

u{x", z") - n(x', z") > ^ (^) n(x", z') - n(x', z') > o, (7) 

U{x", z") - U{x', z") > ^ (^) n(x", z') - n(x', z') > 0. (8) 

^At a first glance, the aggregation property may be less compelling in the context of two-player games. 
However, the results we obtain in this section allows us to cover important examples that are not covered 
by any of our other results. 
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Quasisupermodularity (resp. quasisubmodularity) is sometimes also called the (dual) 
single crossing property (e.g. Milgrom and Shannon, 1994)3 

We say that an aggregative game {X, U) is suhmodular (resp. supermodular) if 11 has 
decreasing (resp. increasing) differences in {x,z) on X x Z. I.e., for all z" > z', x" > x', 

U{x", z") - U{x', z") < {>)Il{x", z') - U{x', z'). (9) 

It is straight-forward to check that if an aggregative game (X, 11) is submodular (resp. 
supermodular), then it is quasisubmodular (resp. quasisupermodular). The converse is 
false. 

A finite aggregative game is quasiconcave (or single-peaked) if for any x, x', x" G X 
with X < x' < x" and 2; G Z, 

Ii{x\ z) > min{n(a;, z), U{x" , z)}. 

A finite aggregative game is quasiconvex if for any x, G X with x < x' < x" and 

zez, 

n(x', z) < max{n(x, 2), n(x", z)}. 

It is strictly quasiconvex if the inequality holds strictly. An action x* E X is a fESS of 
the aggregative game {X, U) if 

n(x*, a(x*, x)) > Il{x, a{x*, x)) for all x E X. 

The following lemma is the key insight for our result on quasiconcave quasisubmodular 
aggregative games. 

Lemma 6 Suppose (X, 11) is a quasiconcave quasisubmodular aggregative game. If x is 
between some x' and a fESS x* , then 

n(x,a(x,x')) > n(x', a(x, x')). 

®It is important to realize that quasisubmodularity in {x,z) where z is the aggregate of all players' 
actions is different from quasisubmodularity in {x,y) where y is the aggregate of all opponents' actions. 
For instance, Schipper (2009, Lemma 1) shows that quasisubmodularity in (x, z) where z is the aggregate 
of all players' actions is satisfied in a Cournot oligopoly if the inverse demand function is decreasing. No 
assumptions on costs are required. It is known from Amir (1996, Theorem 2.1) that further assumptions 
on costs are required if the Cournot oligopoly should be quasisubmodular in (x, y) where y is the aggregate 
of all opponents' actions. 
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Proof. Suppose that x' < x < x* . The case x' > x > x* can be dealt with analogously. 
By the definition of a fESS 

n(x*, a{x\ x)) - n(x', a{x\x')) > 0. 

By quasiconcavity, 

n(x, a(x*, x')) - U{x', a{x*, x')) > 0. 
The result follows then by quasisubmodularity, 

Il{x, a{x, x')) — n(x', a{x, x')) > 0, 
since a{x*,x') < a{x,x'). □ 

Proposition 6 // {X, U) is a finite quasiconcave quasisuhmodular aggregative game for 
which a fESS exists, then imitation is not subject to a money pump. 

Proof. We will show that from any initial action, any relative payoff improving 
sequence of actions reaches a fESS in a finite number of steps. Once reached, there are 
no further improvement possibilities for the maximizer by definition of the fESS. 

Note that since the game is quasiconcave, if x* and x** are fESS, then so is any x G X 
with X* < X < X** or x** < x < x*. We write E for the set of fESS. 

Step 1 : Let yo & X he the starting action of the imitator. Assume that yo < x* = mini? 
(the proof for yo > x** = maxE works analogously). We claim that when the imitator 
switches to a new action yi ^ yo, we must have that yi > yo- Suppose by contradiction 
that yi < yo- By Lemma [T], the imitators would only choose yi if in the previous period 
the maximizer chose x = yi and received a strictly higher payoff than the imitator, 

^{yuVo) = n (yi, a{yi, yo)) - B {yo, a{yi, yo)) > 0. (10) 

But this contradicts Lemma E] as yi < yo < x* . Thus, yi > yo- 

• If yi G E, we are done. 

• U yo < yi < X* , then take yi as the new starting action and repeat Step 1. 

• Else, go to Step 2. 
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Step 2: We have that yi > x**. We claim that when the imitators switches to a new 
action 2/2 7^ yi, we must have that 1/2 < Hi- Suppose by contradiction that 1/2 > Hi- By 
Lemma [H the imitators would only choose ?/2 if in the previous period the maximizer 
chose X = y2 and received a higher payoff, A{y2,yi) > 0. But this contradicts Lemma E] 
as y2> yi> X**. Thus y2 < yi- 

• If ?/2 £ E, we are done. 

• \i yo < y2 < X* , then take y2 as the new starting action and repeat Step 1. 

• If X** < y2 < yi, then take y2 as the new starting action and repeat Step 2. 

We claim that y2 < yo can be ruled out. Since X is finite, the algorithm then stops 
after finite periods. Thus the proof of the proposition is complete once we verify this last 
claim. 

Suppose to the contrary that y2 < yo- By Lemma [H the imitators would only choose 
1/2 if in the previous period the maximizer chose x = y2 and received a strictly higher 
payoff than the imitator, 

A(y2, yi) = n (^2, a{y2, yi)) - n (yi, a{y2, yi)) > 0. 
By quasiconcavity, we have 

n (yo, 0(2/2, yi)) - n {yi, a{y2, yi)) > 0. 
Since a(?/o,yi) > 0(2/2,2/1), we have by quasisubmodularity 

n (2/0, 0(2/0, 2/1)) - n (2/1, 0(2/0, 2/1)) > 0. 
But this contradicts inequality ffTOj) and proves the claim. □ 

The following examples present applications of the previous result. The first exam- 
ple extends the linear Cournot oligopoly of Example [2] to general symmetric Cournot 
oligopoly. 

Example 14 (Cournot Duopoly) Let the symmetric payoff' function be it {x,y) = xp{x+ 
y) — c{x) and assume that 7T{x,y) is quasiconcave in x. S chipper (2009, Lemma 1) shows 
that a symmetric Cournot duopoly with an arbitrary decreasing inverse demand function 
p and arbitrary increasing cost function c is an aggregative quasisubmodular game. Thus, 
Proposition^^ implies that imitation is not subject to a money pump in Cournot duopoly. 
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Example 15 (Rent Seeking) Two contestants compete for a rent v > by bidding 
x,y & X C M^. A player's probability of winning is proportional to her bid, and zero 
if both players bid zero. The cost of bidding equals the bid. The symmetric payoff function 
is given by Tx{x,y) = ^f^'^ — x (see Tullock, 1980, and Hehenkamp, Leininger, and 
Possajennikov, 2004). This game is an aggregative quasisubmodular game (see Schipper, 
2003, Example 6, and Alos-Ferrer and Ania, 2005, Example 2) and 7i{x,y) is concave in 
X. Thus Proposition^^ implies that imitation is not subject to a money pump. 

For quasiconvex quasisupermodular aggregative games we can prove an analogous 
result. We first observe that in a strictly quasiconvex quasisubmodular game a fESS 
must be a "corner" solution if it exists. It follows that there can be at most two fESS. 

Lemma 7 Let (X, 11) be a finite strictly quasiconvex quasisupermodular aggregative game. 
If X* is a fESS, then x* = ma.xX or x* = minX . 

Proof. Let x* be a fESS and suppose to the contrary that there exist x', x" G X such 
that x' < X* < x" . We distinguish four cases: 

Case 1: If 

Il{x\a{x\x")) > n{x',a{x*,x")), 
then by strict quasiconvexity 

U{x*,a{x*,x")) < U{x",a{x\x")), 

a contradiction to x* being a fESS. 

Case 2: The case Il{x' , a{x* , x')) > Il{x" , a{x* , x')) is analogous to Case 1. 
Case 3: If 

U{x',a{x*,x")) > U{x",a{x*,x")), 
then by strict quasiconvexity 

U{x*,a{x*,x")) < U{x',a{x*,x")). 

By quasisupermodularity, 

U{x*,a{x*,x')) < U{x',a{x*,x')). 

a contradiction to x* being a fESS. 

Case 4-' The case Il{x" , a{x* , x')) > Il{x' , a{x* , x')) is analogous to Case 3. 

Thus, if X* is a fESS, then x* = maxX or x* = minX. □ 
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Proposition 7 // {X, U) is a finite strictly quasiconvex quasisupermodular aggregative 
game for which a fESS exists, then imitation is not subject to a money pump. 

Proof. We show that a process of steps that strictly increase the sum of the maxi- 
mizer's relative payoffs must lead to a fESS. Note that only nontrivial steps, in which 
the maximizer does not repeat her action, can improve the sum of her relative payoffs. 
Consider a sequence of nontrivial steps Xi,X2,X3 the maximizer may take. Suppose that 
X2 < Xi (the case X2 > Xi is dealt with analogously). By Lemma [1] it must hold that 

n(x2,a(x2,xi)) > n(xi,a(x2,xi)). (11) 

To show that the process moves to one of the corners, we need to show that either zs > xi 
or X3 < X2. Suppose to the contrary that X2 < X3 < By Lemma [1] it must hold that 

n(x3,a(a;3,a;2)) > n(a;2, 0(^3, X2)). 

Thus, by quasisupermodularity 

n(a;3, a{xi,X2)) > U{x2, a{xi, X2)). (12) 

From inequality f|TT]) follows by strict quasiconvexity that 

n(x2, a{x2, xi)) > n(a;3, a{x2, Xi)), (13) 

with equality only for X3 = Xi. Thus for x^ < Xi, inquality (fT3|) yields a contradiction 
to inequality ( [T2l) . For X3 = xi the sequence of steps has not improved the maximizer's 
sum of relative payoffs since both players obtained the same payoff throughout. 

Thus we have shown that with every nontrivial step, the maximizer gets closer to 
a corner. Since there are only finitely many actions, if the sequence of actions is non- 
constant, then a corner must be reached in finitely many steps. If the corner is a fESS, 
then no further changes of actions occur. Otherwise, the other corner may be reached in 
one additional step. This must be a fESS by Lemma [7] since a fESS is assumed to exist. 
Once it is reached, no further changes of actions occur. □ 



6 Summary and Discussion 

In Table [T] we summarize our results]^ The only class of symmetric games in which 
imitation can really be beaten is the class of games whose relative payoff function is a 

^Thc case of X2 ^ X3 is already excluded by the requirement of non-trivial steps. 

^'^More results on the classes of games and their relationships are contained in our companion paper, 
Duersch, Oechssler, and Schipper (2010). 
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generalized rock-paper-scissors game. While this is a generic class of symmetric games, 
so is its complement. More importantly, many economically relevant games are contained 
in this complement. Thus it is fair to say that imitation seems very hard to beat in large 
classes of economically relevant and generic games. 



Table 1: Summary of Results 



Class 






T*^! Y PI m T» 1 c: 


Symmetric 2x2 games 


essentially unbeatable 


Prop. [2] 


Chicken, Prisoners' Dilemma, 
Stag Hunt 


Additively separable relative 

payoff function 

or 

Relative payoff functions 
with increasing or decreasing 
differences 
or 

1-4 1 ci 1 n 7'*n T^cnr/^TT rrci tvioq inni ri 

exact potential 


essentially unbeatable 
essentially unbeatable 

t-ODClibidili V LLliUCcLLcLUit^ 


Prop. [3] 
Cor.H 

^Ui . lot 


Linear Cournot duopoly 

Heterogeneous Bertrand duopoly 

Public goods 

Common pool resources 

Minimum effort coordination 

Synergistic relationship 

Arms race 

1 ci m n ' o oociTT'Vi 


X V/CilCL IjI V VZ- lJCby\Jl.L tiClilllt,0 Vv 1 Lll 

generalized ordinal potential 




Pron 141 


Examnlerrni 


Quasiconcave relative 
payoff games 


no money pump 


Prop. [5] 


Nash demand game 
Example [TT] 
Example [13] 


Quasiconcave quasisub- 
modular aggregative games 


no money pump 


Prop. M 


Cournot games 
Rent seeking 


Quasiconvex quasisuper- 
modular aggregative games 


no money pump 


Prop. [7] 




No generalized 
Rock-Paper-Scissors games 


no money pump 


Thm. ID 


all of the above 



However, one needs to be aware of the limitations of our analysis, primarily the re- 
striction to two-player games. While a full treatment of the n-player case is beyond 
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the scope of the current paper, we provide here an example that shows how imita- 
tion can be beaten in a standard Cournot game when there are three players. Let the 
inverse demand function be p{Q) = 100 — Q and the cost function be c{qi) = lOgj. 
Now consider the case of two maximizers and one imitator. Writing a vector of quan- 
tities as (<?/, (?M, ^m), it is easy to check that the following sequence of action profiles 
(0,22.5, 22.5), (22.5, 0,68), (0,22.5, 22.5), (22.5, 68,0), (0,22.5, 22.5)... is an imitation cy- 
cle. The two maximizers take turns in inducing the imitator to reduce his quantity to 
zero by increasing quantity so much that price is below marginal cost. Since the other 
maximizer has zero losses, she is imitated in the next period, which yields half of the 
monopoly profit for both maximizers. Clearly, this requires coordination among the two 
maximizers but this can be achieved in an infinitely repeated game by the use of a trigger 
strategy. Thus, imitation is subject to a money pump. Recall, however, that we pitted 
imitation against truly sophisticated opponents. Whether imitation can be beaten also 
by less sophisticated (e.g. human) opponents remains to be seen in future experiments. 

Furthermore, our analysis was based on the assumption that an imitator sticks to his 
action in case of a tie in payoffs. To see what goes wrong with an alternative tie-braking 
rule consider a homogenous Bertrand duopoly with constant marginal costs. Suppose 
the imitator starts with a price equal to marginal cost. If the maximizer chooses a 
price strictly above marginal cost, her profit is also zero. If nevertheless, the maximizer 
were imitated, she could start the money pump by undercutting the imitator until they 
reach again price equal to marginal cost and then start the cycle again. This example 
shows that our results depend crucially on the details of the imitation heuristics. It 
would be interesting to exactly characterize the class of simple decision heuristics that 
are essentially unbeatable in large classes of economically relevant games. We leave this 
for future work. 
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